1. Technical Field
This invention relates to a decoding system and method for proper interpolation for motion compensation.
2. Background Art
In some applications, a digital video data stream needs to be displayed at a smaller resolution than it has upon reception. For example, as high definition television (HDTV) is likely to become the digital TV broadcast standard in the U.S., there is a need for low cost decoding systems with High Definition (HD) capacity but Standard Definition (SD)-format output. In a standard MPEG-2 decoding system, for example, three frames of memory are needed for use in decoding the input stream, one for backward reference, one for forward reference, and a third one for the current frame. However, the frame memory size is matched to input resolution, i.e., if input is HD, 3 frames of HD size memory are required to decode the input stream.
An external scaler could be added to such a standard MPEG-2 decoding system to reduce the output resolution. However, the cost for such a system is HD resolution frame memory, HD resolution decoding complexity, and spatial (pixel) domain filtering for down scaling. Alternatively, by downscaling the reference frame just before storage in such a standard MPEG-2 decoding system, memory can be saved by matching memory requirement to the output resolution (SD resolution frame memory can be provided). However, there is no saving regarding computation complexity in this approach, since the decoding loop is still working at full (input) resolution. Furthermore, upscaling would have to be added before motion compensation (motion compensation), which further increases the computation complexity. The downscaling can be moved further forward in the decoding path so that the motion compensation can work in the reduced resolution as well, i.e., no upscaling is needed. Motion vectors, in this case, are needed to be scaled down for the reduced resolution motion compensation. As the motion vectors are scaled down, their precision increase. For a downscaling factor of 2, for example, the motion vectors after scaling are half in magnitude but twice in precision (from ½ pel to ¼ pel.)
This increase in motion vector precision results in more cases where interpolation is required (i.e., when the motion vector is non-integer). However, the interpolation in the reduced resolution reference frame can cause problems when the frame is interlaced. MPEG-2, for example, supports both field motion compensation and frame motion compensation for interlaced digital video as illustrated in FIG. 1. Referring to FIG. 1, in field motion compensation, data is fetched within separate top and bottom fields and then interlaced to construct a macroblock. It is straightforward to interpolate within a field if field motion compensation is used. In frame motion compensation, however, data is fetched from a frame comprised of a mixture of top and bottom fields to construct a macroblock. Locally, however, the data area to be fetched from the frame might be either stationary/progressive or moving/interlaced. Although generally frame motion compensation is used in progressive data areas and frame-based interpolation applies, it is not always so. Frame motion compensation could be used in interlaced data areas, as long as the encoding process has a good reason to believe this is the best way to estimate the current macroblock. If this is the case, frame-based interpolation can cause the problem of fetching the wrong field, and therefore, interpolation should be done on a field basis for frame motion compensation.
Unfortunately, whenever frame motion compensation is used standard decoding systems cannot determine locally whether the data area to be fetched contains progressive or interlaced data. Furthermore, although there are ways to detect vertical high frequencies, to do such detection in the spatial domain is not trivial in terms of computation, and the presence of vertical high frequencies is neither a sufficient nor a necessary condition for indicating interlacing. That is, not all high vertical frequencies result from interlacing, as there are high vertical frequencies in still areas as well.
Thus, whenever frame motion compensation is used existing decoding systems and methods choose only one mode of interpolation (field-based or frame-based) off line and apply it blindly to the whole picture. Both modes of interpolation, however, can cause problems when blindly applied. Specifically, if the frame-based interpolation is applied, it risks running into the problem of fetching the wrong fields in the interlaced moving area (the visual effect is block artifact). On the other hand, field-based interpolation faces the problem of bluriness and aliasing in the progressive stationary area.
Accordingly, what is needed is intelligent video decoding (including scalable video decoding) for video output (including lower-resolution video output) that can dynamically in real time apply proper interpolation according to the local nature of the data area to be fetched whenever frame motion compensation is used. Also what is needed is a decoding system for such intelligent video decoding that has SD resolution frame memory, SD resolution decoding complexity, and no spatial domain filtering. The present invention can solve the previously mentioned drawbacks of prior decoding systems and methods through various embodiments, including a decoding system and method for proper interpolation for reduced resolution motion compensation.